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1. (Currently Amended) A reliable symbol identification method for use in a communication 
system for transmitting symbols of a high order constellation, comprising : 

calculating a reliability factor of a captured sample from values of a plurality of other 
samples in proximity to the captured sample, wherein the captured sample and the plurality of 
other samples represent a data signal recovered from a communication channel, and 

if the reliability factor is less than a predetermined limit, designating the captured 
sample as a reliable symbol. 

2. (Previously Presented) The method of claim 1, wherein the reliability factor R„ of the 
captured sample is given by: 



y n is the captured sample, 

y n _i is a sample in proximity to the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficient representing any prior knowledge of intersymbol interference effects . 

3. (Original) The method of claim 2, where q = 1 for all i. 

4. (Original) The method of claim 2, wherein Ki = 0. 

5. (Original) The method of claim 2, wherein K 2 = 0. 

6. (Currently Amended) The method of claim 1, wherein the predetermined limit varies 
over time. 

7. (Currently Amended) The method of claim 1, further comprising determining a rate at 
which reliable symbols are identified, and 

if the rate is less than a predetermined value, increasing the predetermined limit. 



K, 

Rn= l|y n -ii*Ci, where 



8. (Currently Amended) The method of claim 1, further comprising determining a rate at 
which reliable symbols are identified, and 
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if the rate exceeds a second predetermined value, decreasing the predetermined limit. 

9. (Previously Presented) The method of claim 1, wherein the reliability of a two- 
dimensional captured sample y n is given by: 



in first and second dimensions, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficient representing anv prior knowledge of intersvmbol interference effects . 

10. (Currently Amended) A method of identifying reliable symbols for use in a 
communication system for transmitting symbols of a high order constellation , comprising:; 

for a captured sample y n recovered from a communication channel: 

iteratively, for i= -Ki to K 2 , i*0, wherein K u K 2 are real numbers: 

adding to a reliability factor a value of based on another captured sample' 
y n -i also recovered from the communication channel, 

if the reliability factor exceeds a predetermined limit, disqualifying the 
captured sample as a reliable symbol, and 

otherwise, incrementing i and, if i=0, re-incrementing i for a subsequent 
iteration: and 

thereafter, unless the captured symbol has been disqualified, designating the" 
captured sample as a reliable symbol. 

11. (Previously Presented) The method of claim 10, wherein the adding adds an absolute 
value of the sample y n -i to the reliability factor. 

12. (Previously Presented) The method of claim 10, wherein the adding adds a scaled value 
of the sample y n -j to the reliability factor, the value scaled in accordance with a predeterm i ned 
coefficient c representing anv prior knowledge of intersymbol interference effects . 

13. (Previously Presented) The method of claim 10, wherein the adding adds the power of 
the sample y n -i to the reliability factor. 
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14. (Previously Presented) The method of claim 10, wherein the predetermined limit is half a 
width of an annular constellation ring in which the captured sample is observed. 

15. (Previously Presented) The method of claim 10, wherein the predetermined limit is 
+K 2 )d m j n where dm in is half a distance between two constellation points that are closest 

together in a governing constellation. 

16. (Previously Presented) The method of claim 10, wherein the predetermined limit varies 
over time. 

17. (Previously Presented) The method of claim 10, further comprising determining a rate 
at which reliable symbols are identified, and 

if the rate is less than a predetermined value, increasing the predetermined limit. 

18. (Previously Presented) The method of claim 10, further comprising determining a rate 
at which reliable symbols are identified, and 

if the rate exceeds a second predetermined value, decreasing the predetermined limit. 

19. (Currently Amended) A method of identifying reliable symbols , for use in a 
communication system for transmitting symbols of a high order constellation, comprising!.; 

for a captured sample recovered from a communication channel, 

determining whether any value of a plurality of neighboring samples also' 
recovered from the communication channel is within a predetermined limit and 

if none of the values exceed the predetermined limit, designating the captured 
sample as a reliable symbol. 

20. (Original) The method of claim 19, wherein the predetermined limit varies over time. 

21. (Original) The method of claim 19, further comprising determining a rate at which 
reliable symbols are identified, 

if the rate is less than a predetermined threshold, increasing the predetermined limit. 

22. (Original) The method of claim 21, further comprising, if the rate exceeds a second 
predetermined threshold, decreasing the predetermined limit. 
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23. (Currently Amended) The method of claim 19, wherein the plurality of neighboring 
samples occur in a first window adjacent to the captured sample on one side of the captured 
sample. 

24. (Currently Amended) The method of claim 19, wherein the plurality of neighboring 
samples occur in a pair of windows that are adjacent to, and on either side of the captured 
sample. 

25. (Currently Amended) A method of detecting reliable symbols within a sampled data 
signal,, for use at a receiver of a communication system for transmitting symbols of a high order 
constellation, comprising : 

identifying a sequence of sample values having values within a predetermined limit, and 
designating a sample adjacent to the sequence as a reliable symbol. 

26. (Original) The method of claim 25, wherein the predetermined limit varies over time. 

27. (Original) The method of claim 25, further comprising determining a rate at which 
reliable symbols are identified, 

if the rate is less than a predetermined threshold, increasing the predetermined limit. 

28. (Original) The method of claim 27, further comprising, if the rate exceeds a second 
predetermined threshold, decreasing the predetermined limit. 

29. (Previously Presented) A data decoder for use in a communication system for 
transmitting symbols of a high order constellation, comprising: 

a reliable symbol detector to detect reliable symbols from a sequence of captured 
samples, the reliable symbols being the captured samples which are estimated to be located in 
a correct decision region of a corresponding source symbol, have been corrupt e d l east by 
i ntcrgymbo l i nterference ("ISI"), 

an adaptation unit coupled to the reliable symbol detector to generate intersvmbol 
interference H SH metrics based on the reliable symbols, and 

a data decoder to receive the captured samples and estimated source symbols based on 
the ISI metrics. 
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30. (Currently Amended) An equalization method for use in a communication system for 
transmitting symbols of a high order constellation, comprising 

identifying reliable symbols from a string of captured samples recovered from a 
communication channel, the reliable symbols being the captured samples which are estimated 
to be located in a correct decision region of their corresponding source symbols, have been 
corrupted least by effects of the commun i cat i on channel, 

calculating the-channel effects based on the reliable symbols and samples adjacent 
thereto, 

correcting the captured samples based on the calculated channel effects to equalize the 
string of captured samples. 

31. (Currently Amended) The method of claim 30, wherein the identifying comprises: 
calculating a reliability factor of a captured sample from values of a plurality of samples 

in the neighborhood of the captured sample, 

if the reliability factor is below a predetermined limit, designating the captured sample 
as a reliable symbol. 

32. (Currently Amended) The method of claim 31, wherein the reliability factor of the 
captured sample y n is given by: 



y n -i is a sample in the neighborhood of the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficien t representing any prior knowledge of intersymbol interference effects . 

33. (Currently Amended) The method of claim 31, wherein the reliability factor of the 
captured sample y n is given by: 



y n .j is a sample in the neighborhood of the captured sample, 
K is a length of samples, and 

q is a coefficient representing any prior knowledge of intersymbol interference effects . 



Rn= &y n -i|-Ci, where 



Rn = Z|y n -i|-C|, where 



i=1 
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34. (Currently Amended) The method of claim 31, wherein the reliability of a two- 
dimensional captured sample y n is given by: 



in first and second dimensions, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficien t representing any prior knowledge of intersymbol interference effects . 

35. (Original) The method of claim 30, wherein the identifying comprises: 

identifying a sequence of samples having received signal magnitude levels below a 
predetermined limit, and 

designating a sample adjacent to the sequence as a reliable symbol. 

36. (Original) The method of claim 30, wherein, for QAM transmission, the identifying 
comprises: 

identifying a sequence of samples for which a received signal magnitude in a 
quadrature-phase component is below a predetermined limit, and 

designating an adjacent sample as a reliable symbol for quadrature-phase. 

37. (Original) The method of claim 30, wherein, for QAM transmission, the identifying 
comprises: 

identifying a sequence of samples for which a received signal magnitude in an in-phase 
component is below a predetermined limit, and 

designating an adjacent sample as a reliable symbol for in-phase. 

38. (Original) The method of claim 30, wherein the calculating estimates K channel 

K 

coefficients ai according to a least squared error analysis of y RS -x n - Y,a\*n-\ » solving for aj, 



for a plurality of reliable symbols yRs, where x n and x n _j are estimated transmitted symbols. 

39. (Original) The method of claim 30, further comprising assigning weights among the 
reliable symbols based upon respective reliability factors. 




yi and v, _-y^—aftd--y|— respectively represent values of a neighboring sample y n -i 
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40. (Previously Presented) An equalizer , for use in a communication system for transmitting 
symbols of a high order constellation, comprising: 

a buffer memory, 

a reliable symbol detector in communication with the buffer memory, the detector to 
estimate which samples from a sequence of captured samples are located in a correct decision 
region of their corresponding source symbols, have been corrupted I caGt by channel effects, 

an adaptation unit in communication with the reliable symbol detector to estimate 
channel effects based on the values of the reliable symbols and samples adjacent thereto, and 

a symbol decoder in communication with the adaptation unit and the buffer memory. 

41. (Currently Amended) The equalizer of claim 40, wherein the reliable symbol operates 
according to a method, comprising: 

calculating a reliability factor of a captured sample from values of a plurality of samples 
proximate to the captured sample, and 

if the reliability factor is less than a predetermined limit, designating the captured 
sample as a reliable symbol. 

42. (Currently Amended) The equalizer of claim 41, wherein the reliability factor Rn of the 
captured sample is given by: 



y n is the captured sample, 

y n -j is a sample in proximity to the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

Cj is a coefficient representing any prior knowledge of intersvmbol interference effects . 

43. (Currently Amended) The equalizer of claim 41, wherein the reliability of a two- 
dimensional captured sample y n is given by: 




1=^ 

i*0 




■afld--y|— -respectively represent values of a neighboring sample y„-i 



in first and second dimensions, 
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Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficien t representing any prior knowledge of intersymbol interference effects . 

44. (Previously Presented) A receiver , for use in a communication system for transmitting 
symbols of a high order constellation, comprising: 

a demodulator to sample and capture transmitted data from a channel, 
a buffer memory in communication with the demodulator to store values of captured 
samples, 

a processor executing instructions that establish the following logical structures therein: 

a reliable symbol detector in communication with the buffer memory to identify 
which of the stored captured samples are likely to be located in a correct decision region 
of a corresponding source symbol, hove been corrupted l east by chann el e ff e cts, the 
identified samples being reliable symbols, 

an adaptation unit in communication with the reliable symbol detector to 
estimate channel effects from values of the reliable symbols, and 
a symbol decoder unit in communication with the adaptation unit and the buffer 
memory. 

45. (Currently Amended) The receiver of claim 44, wherein the reliable symbol detector 
operates according to a method, comprising: 

calculating a reliability factor of a captured sample from values of a plurality of samples 
proximate to the captured sample, and 

if the reliability factor is less than a predetermined limit, designating the captured 
sample as a reliable symbol. 

46. (Currently Amended) The receiver of claim 45, wherein the reliability factor Rn of the 
captured sample is given by: 



y n is the captured sample, 

y n -i is a sample in proximity to the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficient representing any prior knowledge of intersymbol interference effects . 
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47. (Currently Amended) The receiver of claim 45, wherein the reliability of a two- 
dimensional captured sample y n is given by: 



in first and second dimensions, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

d is a coefficient representing any prior knowledge of intersvmbol interference effects . 

48. (Original) The receiver of claim 44, further comprising a second buffer memory in 
communication with the symbol decoder. 

49. (Previously Presented) A transmi s sion data communication system comprising: 

a source that transmits data encoded as symbols, the symbols being selected from a 
high-order constellation, 

a destination that captures a signal representing the transmitted symbols having been 
corrupted by at least intersymbol interference, the destination: 

identifying reliable symbols from the captured samples, reliable symbols being 
those captured samples that are estimated to be located in a correct decision region of 
their corresponding source symbols, corrupted l east by inter s ymbo l i nterf ere nc e , 

calculating channel effects based on the reliable symbols and samples proximate 
thereto, and 



50. (Currently Amended) The system of claim 49, wherein reliable symbols are identified 
according to a method comprising: 

calculating a reliability factor of a captured sample from values of a plurality of samples 
proximate to the captured sample, and 

if the reliability factor is less than a predetermined limit, designating the captured 
sample as a reliable symbol. 

51. (Currently Amended) The system of claim 50, wherein the reliability factor Rn of the 
captured sample is given by: 




y ln ,jind_y 2n .--yf 



-af^--y|— -respectively represent values of a neighboring sample y n -i 



correcting other captured samples based on the channel effects. 
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Rn= 5|yn-i|-Ci, where 

i = -K, 
i*0 



y n is the captured sample, 

y n .j is a sample in proximity to the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

d is a coefficient representing any prior knowledge of intersymbol interference effects . 

52. (Currently Amended) The system of claim 50, wherein the reliability of a two- 
dimensional captured sample y n is given by: 



in first and second dimensions, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficien t representing any prior knowledge of intersvmbol interference effects . 

53. (Currently Amended) A computer readable medium having stored thereon instructions 
that, when executed, cause a processor to identify reliable symbols from captured samples 
received bv a system for use in communicating data via a high order constellation by a process 
comprising : 

calculatjng_e-a reliability factor of a captured sample from values of a plurality of 
samples proximate to the captured sample, and 

if the reliability factor is less than a predetermined limit, designa ting e-the captured 
sample as a reliable symbol. 

54. (Currently Amended) The medium of claim 53, wherein the reliability factor Rn of the 
captured sample is given by: 




k. 



r n-i 



K, 

Rn= Z|yn-i|-Ci, where 



i=-Ki 
i*0 



y n is the captured sample, 

y n .j is a sample in proximity to the captured sample, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 
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q is a coefficient representing any prior knowledge of intersvmbol interference effects . 

55. (Currently Amended) The medium of claim 53, wherein the reliability of a two- 
dimensional captured sample y n is given by: 

Rn=. £ 7 y i 2 n-i +y L c i' where 

Yi and y, _-y?—-aftd-y?— -respectively represent values of a neighboring sample y n -i 

in first and second dimensions, 

Ki, K 2 are numbers of samples adjacent to the captured sample, and 

q is a coefficient representing any prior knowledge of intersymbol interference effects . 

56. (Previously Presented) A computer readable medium having stored thereon instructions 
that, when executed, cause a processor to correct channel effects in captured samples received 
by a system for use in communicating data via a high order constellation bv a process 
comprising : 

identifying reliable symbols from a string of captured samples, the reliable symbols 
being the captured samples which are estimated to be located in a correct decision region of a 
corresponding source symbol, have been corrupted least by channe l effects, 

calculatinge channel effects based on the reliable symbols and samples proximate 
thereto, and 

correcting the captured samples based on the channel effects. 

57. (Currently Amended) A method^ for use in a communication system for transmitting 
symbols of a high order constellation, of decoding a string of captured samples recovered from 
a communication channel comprising: 

identifying reliable symbols from a the string of captured samples, the reliable symbols 
being the captured samples which are estimated to be located in a correct decision region of a 
corresponding source symbol, hove been corrupted l east by channe l effects, 

calculating channel effects based on the reliable symbols and samples proximate 
thereto, 

estimating transmitted symbols from remaining captured samples based on the channel 
effects, and 

outputting the estimated symbols as the a decoded data signal. 
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